---
title: "Short-term impact of a malaria elimination campaign in Southern Mozambique"
output: 
  flexdashboard::flex_dashboard:
    theme: yeti
    storyboard: true
    social: menu
    source: embed
---

```{r setup, include=FALSE}
library(flexdashboard)
```

```{r, echo = FALSE, warning = FALSE, message = FALSE, comment = NA, error= FALSE, cache = F}
library(knitr)
opts_chunk$set(comment = NA, 
               echo = FALSE, 
               warning = FALSE, 
               message = FALSE, 
               error = TRUE, 
               cache = FALSE)
```


```{r, results = 'hide'}
# Packages
library(ggplot2)
library(cism)
library(rgeos)
library(maptools)
library(rgdal)
library(tidyr)
library(RColorBrewer)
library(dplyr)
library(leaflet)
library(readr)
library(ggthemes)
library(ggrepel)
library(knitr)
library(kableExtra)
options(knitr.table.format = "html") 
# Define a theme
theme_maltem <- ggthemes::theme_hc
```


```{r}
if('presentation_data.RData' %in% dir()){
  load('presentation_data.RData')
} else {
  source('../../master.R', chdir = TRUE)

  tablify <- function(x, n = 5){
    DT::datatable(x,
                  selection = 'none',
                  escape = FALSE,
                  options = list(#sDom  = '<"top">lrt<"bottom">ip',
                                 pageLength = n, 
                                 dom = 'tip'))
  }
  kablify <- function(x, size = 16){
    kable(x) %>% 
    kable_styling(bootstrap_options = c("striped",
    "hover",
    "condensed"),
    font_size = size,
    position = 'float_left')
  }
  save.image('presentation_data.RData')
}

```





### Hello


```{r}
# africa <- cism::africa
# africa_small <- thinnedSpatialPoly(SP = africa,
#                                      minarea = 0,
#                                      tolerance = 20,
#                                      topologyPreserve = TRUE)
# cols <- colorRampPalette(brewer.pal(9, 'Blues')[3:9])(nrow(africa_small))
# cols <- sample(cols)
# cols <- adjustcolor(cols, alpha.f = 0.8)
# plot(africa_small, col = cols, border = NA)

ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'How effective is a malaria\nelimination campaign in Mozambique?'),
            size = 8)
```




***


- Joe Brew  
- joe@databrew.cc  
- [economicsofmalaria.com](http://economicsofmalaria.com) 
- [economicsofmalaria.com/tgh](http://economicsofmalaria.com/tgh) 


###  The burden of malaria

Intro




```{r}
options(scipen = '999')
x <- data_frame(year = c(2000, 2015),
                deaths = c(839000, 438000))
g1 <- 
  ggplot(data = x,
       aes(x = year,
           y = deaths)) +
  geom_area(fill = 'darkorange',
            alpha = 0.6) +
  geom_point(alpha = 0.6,
             size = 2,
             color = 'blue') +
  theme_maltem() +
  labs(x = 'Year',
       y = 'Deaths',
       title = 'How many people die?')

y <- data_frame(region = c('Africa', 'Everywhere else'),
                percent = c(90, 10))
g2 <- 
  ggplot(data = y,
       aes(x = region,
           y = percent)) +
  geom_bar(stat = 'identity',
           fill = 'darkorange', 
           alpha = 0.6) +
  theme_maltem() +
  labs(x = 'Region',
       y = 'Percent',
       title = 'Where do they die?')
Rmisc::multiplot(g1, g2, cols = 2)
```

***

- 2015: 212M cases; 429K deaths (global). 
- Mozambique among ten highest burden countries in world. 
- 29% reduction since 2010.  
- Now, focus on _elimination_ and _eradication_.

###  How to eliminate

Intro


```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = '?'),
            size = 100)
```

***

- Scale up vs. innovate.  
- Mass drug administration (MDA).  
- Other, more time-tested strategies (ITNs, IRS).
- The need for operational research. 

###  Challenges in evaluation

Intro

```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'Evaluation != Easy'),
            size = 13)
```

***

- Difficult to measure impact of interventions in operational research.  
- OR is usually not experimental, not randomized, not controlled.
- Before vs. after and case vs. control approaches are useful, but biased.


###  Why evaluate

Intro


```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'Evaluation = Necessary'),
            size = 13)
```

***

- WHO: "the need for more research on the optimum methods of implementing MDA programs, promoting community participation and compliance with treatment, and evaluating their effectiveness.""

- Sustainability and cost-effectiveness are among the main concerns of funders, stakeholders and policymakers.

- Evidence of the causal impact of malaria elimination initiatives is a pre-requisite for evaluating their cost-effectiveness. 


###  How to evaluate

Intro


```{r}
ggplot() +
  theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'We need a\nCOUNTERFACTUAL'),
            size = 13)
```

***

- Since we don't have an experiment, we need quasi-experimental approaches.  
- The synthetic control method (SCM).  
- We want to construct a _realistic_ counterfactual telling us _what would have happened_ without the campaign.  




###  What are we evaluating

Intro


```{r}
cols <- c('darkorange', 'darkgreen')
moz2_fortified <- moz2_fortified
map <- moz2_fortified %>%
  filter(id %in% c('Manhiça',
                   'Magude'))
ggplot() +
  geom_polygon(data = map,
               aes(x = long,
                   y = lat,
                   group = id,
                   fill = id),
               color = 'white',
               # color = 'grey',
               alpha = 0.8) +
  theme_map() +
  scale_fill_manual(name = '',
                    values = cols) +
  # theme_black(base_size = 0) +
  theme(axis.line = element_blank(), axis.text = element_blank(),
        axis.ticks = element_blank(), axis.title = element_blank(),
        legend.position = 'none') 
```

***

- Malaria elimination campaign in Magude.  
- Deployed during rainy seasons of 2015-6 and 2016-7. 
- Consisted of MDA, IRS, and surveillance. 
- ITN coverage was already high from 2014. 


###  The data

Methods



```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  group_by(date, district) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

ggplot(data = x,
       aes(x = date,
           y = p)) +
  geom_line(alpha = 0.8,
            color = 'darkred') +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Weekly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Weekly incidence: all districts',
       subtitle = 'All age groups') +
  theme_maltem() +
  facet_wrap(~district) +
  theme(legend.position="none") +
  theme(strip.background = element_rect(fill="white"),
        strip.text = element_text(color = grey(0.1), size = 6)) +
  theme(axis.text.x = element_text(angle = 90))
```

***

- Mozambique’s Boletim Epidemiológico Semanal (BES) of the NMCP.  
- Weekly clinical cases, by age group, by health facility, for each of a few high risk diseases
, the epidemic disease reporting system used by the NMCP. 
- Population estimates from the National Statistical Institute (INE). 

###  Identification strategy and variables


Methods



```{r}
ggplot() + theme_void() +
  geom_text(aes(x = 0, y = 0, label = 'analysis variables:\n
- LLINs coverage\n    
- IRS coverage\n  
- Average weekly temperature\n
- Weekly rainfall'),
            size = 7)
```



###  Rainfall


Methods


```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(place = factor(place, levels = c('Other districts', 'Magude'))) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, district, place) %>%
  summarise(precipitation = mean(precipitation),
            temp = mean(temp),
            temp_max = max(temp_max),
            temp_min = min(temp_min))

g <- ggplot() +
  geom_line(alpha = 0.2,
            data = x %>% filter(place == 'Other districts'),
       aes(x = date,
           y = precipitation,
           group = district)) +
    geom_line(alpha = 0.8,
              color = 'darkred',
            data = x %>% filter(place == 'Magude'),
       aes(x = date,
           y = precipitation,
           group = district)) +
  labs(x = 'Date',
       y = 'Milimeters',
       title = 'Precipitation: Magude vs. other districts',
       subtitle = 'Monthly average precipitation, Magude in red') +
  theme_maltem()
g 
```

***

(Pretty similar everywhere)


###  Temperature


Methods


```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(place = factor(place, levels = c('Other districts', 'Magude'))) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, district, place) %>%
  summarise(precipitation = mean(precipitation),
            temp = mean(temp),
            temp_max = max(temp_max),
            temp_min = min(temp_min))

ggplot() +
  geom_line(alpha = 0.2,
            data = x %>% filter(place == 'Other districts'),
       aes(x = date,
           y = temp,
           group = district)) +
    geom_line(alpha = 0.8,
              color = 'darkred',
            data = x %>% filter(place == 'Magude'),
       aes(x = date,
           y = temp,
           group = district)) +
  labs(x = 'Date',
       y = 'Degrees (celcius)',
       title = 'Average temperature: Magude vs. other districts',
       subtitle = 'Monthly average, Magude in red') +
  theme_maltem() +
  xlim(as.Date('2014-01-01'),
               max(x$date))
```

***


(Pretty similar everywhere)


###  Difference in differences


Methods


```{r}
a <- seq(1, 5, length = 10)
b<- c(seq(2, 4, length = 5), seq(5, 7, length = 5))
x <- data_frame(time = 1:10,
                 a,
                 b)
x <- tidyr::gather(x, key, value, a:b)
x <- x %>%
  mutate(key = ifelse(key == 'a', 'Control', 'Intervention'))
ggplot(data = x,
       aes(x = time,
           y = value,
           color = key)) +
  geom_line() +
  geom_point() +
  theme_void() +
  scale_color_manual(name = '',
                     values = c('red', 'blue')) +
  geom_vline(xintercept = 5,
             alpha = 0.6,
             lty = 2)
```

***

- We _could_ compare changes in malaria incidence to other districts using a DD model. 
- Often used to estimate treatment effects in such settings by contrasting changes in outcome pre- and -post intervention, for the treatment and control groups.




###  Why we can't use difference in differences


Methods


```{r}
x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, place) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

g1 <- ggplot(data = x,
       aes(x = date,
           y = p,
           group = place)) +
  geom_line(aes(color = place), alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Monthly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Incidence: Magude vs. other districts',
       subtitle = 'All age groups') +
  theme_maltem() +
  xlim(as.Date('2013-01-01'), max(x$date)) +
  geom_vline(xintercept = as.numeric(as.Date('2015-10-01')))


x2 <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  mutate(date = as.Date(paste0(year, '-', month, '-01'))) %>%
  group_by(date, place) %>%
  summarise(p = sum(cases) / sum(population) * 1000) %>%
  group_by(date) %>%
  summarise(magude_to_other = p[place == 'Magude'] / 
              p[place == 'Other districts'] * 100)

g2 <- ggplot(data = x2,
       aes(x = date,
           y = magude_to_other)) +
  geom_line(alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  geom_hline(yintercept = 100,
             col = 'darkred',
             alpha = 0.6,
             lty = 2) +
  labs(x = 'Date',
       y = 'Percent') +
  labs(title = 'Magude incidence as % of other districts',
       subtitle = 'All age groups') +
  theme_maltem() +
  xlim(as.Date('2013-01-01'), max(x$date))
g1
# Rmisc::multiplot(g1, g2, cols = 2)
```


***

- Magude shows a downward trend in incidence over time, even prior to the elimination campaign.
- Lack of a visible parallel trend between treatment and control groups 


###  Synthetic control model


Methods


```{r}
set.seed(500)
cols <- c('darkorange', 'darkgreen')
moz2_fortified <- moz2_fortified
map <- moz2_fortified %>%
  filter(id %in% c('Manhiça',
                   'Magude'))

centroids <- moz3
centroids <- centroids[centroids@data$NAME_2 %in% c('Manhiça', 'Magude'),]
centroids@data$color <- ifelse(centroids@data$NAME_2 == 'Magude',
                               'red', 
                               'white')
coords <- coordinates(centroids)
coords <- data.frame(coords)
names(coords) <- c('x', 'y')
centroids@data <- cbind(centroids@data, coords)
centroids$size <- rnorm(mean = 2, n = nrow(centroids))
centroids$size <- ifelse(centroids@data$NAME_3 == 'Magude',
                         0, 
                         centroids$size)
centroids$size <- centroids$size^10
ggplot() +
  geom_polygon(data = map,
               aes(x = long,
                   y = lat,
                   group = id,
                   fill = id),
               color = 'white',
               # color = 'grey',
               alpha = 0.8) +
  theme_map() +
  scale_fill_manual(name = '',
                    values = cols) +
  # theme_black(base_size = 0) +
  theme(axis.line = element_blank(), axis.text = element_blank(),
        axis.ticks = element_blank(), axis.title = element_blank(),
        legend.position = 'none') +
  geom_point(data = centroids@data,
             aes(x = x,
                 y = y,
                 size = size),
             alpha = 0.6,
             color = 'darkred')
```

***

- We employ the synthetic control method (SCM).  
- Weighted combination of potential control districts, the synthetic control, using the districts that best approximate the most relevant characteristics of malaria incidence in Magude before the intervention. 


### Magude vs. other districts

Results


```{r}

x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  group_by(date, place) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

ggplot(data = x,
       aes(x = date,
           y = p,
           group = place)) +
  geom_line(aes(color = place), alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Weekly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Weekly incidence: Magude vs. other districts',
       subtitle = 'All age groups') +
  theme_maltem()
```

***

Incidence, not adjusted for weighted contributions.

### Magude vs. other districts by age

Results


```{r}

x <- df %>%
  filter(disease == 'MALARIA') %>%
  mutate(place = ifelse(district == 'MAGUDE', 'Magude',  'Other districts')) %>%
  group_by(date, place, age_group) %>%
  summarise(p = sum(cases) / sum(population) * 1000)

ggplot(data = x,
       aes(x = date,
           y = p,
           group = place)) +
  geom_line(aes(color = place), alpha = 0.8) +
  geom_vline(xintercept = as.numeric(as.Date(paste0(2010:2017, '-01-01'))),
             color = 'black',
             alpha = 0.3) +
  labs(x = 'Date',
       y = 'Weekly cases per 1,000') +
  scale_color_manual(name = 'District',
                     values = c('red', 'black')) +
  labs(title = 'Weekly incidence: Magude vs. other districts',
       subtitle = 'Segregated by age group') +
  theme_maltem() +
  facet_wrap(~age_group)
```

***

Effect is clearer in younger children.


### 0-4 year-olds: Magude vs synthetic control  

Results

 






### 5+ year-olds: Magude vs synthetic control  

Results
 
 


### Placebo testing for diarrhea

Results








### Effect of interevention

Results






***

Relative to the synthetic control, the average treatment effect during the first post-intervention malaria season was a reduction in weekly incidence by 4 and 2 cases per 1,000 in those aged 0-4 and 5 or older, respectively. During peak malaria season (January - March), 1,939 cases were prevented in 2016, and 3,405 in 2017. 


### Our findings

Results



```{r}
x <- data_frame(year = 2015:2017,
                value = c(100, 100 - c(91.29, 81.3)))
ggplot(data = x,
       aes(x = year,
           y = value)) +
  geom_line() +
  geom_point(size = 20,
             alpha = 0.5) +
  theme_void() +
  geom_hline(yintercept = c(0, 100),
             lty = 2,
             alpha = 0.6) +
    geom_label(aes(label = round(value))) 
```

***

-  Across age groups during the peak malaria season (January-March) incidence declined by 91.29% and 81.3% in 2016 and 2017, respectively.  
- Has not achieved zero cases.  
- Unsure, at this point, to what extent we can attribute cases to importation or other causes. 

### Implications  


Discussion


- The strategy reduced malaria significantly.   
- Effectiveness quantified accurately enough to allow for granular cost-effectiveness.   
- Synthetic control method useful and generalizable for the estimation of similar initiatives’ effectiveness.   
- Policy-makers, researchers, and those actively engaged in or planning disease elimination initiatives, can benefit from both the methods and findings from this study.


### Thank you


Fin

[http://www.economicsofmalaria.com](economicsofmalaria.com)  
joe@databrew.cc  


### Extra slides  

Supplementary slides begin here.  


### The synthetic control

Results

```{r}
# Copy of Ranjeeta's table
x <- dplyr::data_frame(Indicator = c('Incidence, rainy season 2015',
                              'Incidence, rainy season 2014', 
                              'LLIN coverage',
                              'Pre-intervention precipitation',
                              'Pre-intervention temperature',
                              'Post-intervention precipitation'),
                `Magude 0-4` = c(3.65, 8.87, 24.82, 1.71, 24.52, 1.84),
                `Synthetic Magude 0-4` = c(5.1, 6.93, 24.77, 1.74, 24.51, 1.82),
                `Magude 5+` = c(4.02, 8.53, 24.83, 1.7, 24.52, 1.84),
                `Synthetic Magude 5+` = c(4.12, 6.29, 25.22, 1.71, 24.49, 1.83))
knitr::kable(x, format = 'html')

```

***


```{r}
uw <-
  df %>%
  filter(age_group == '0-4') %>%
  filter(disease == 'MALARIA',
         district != 'XAI-XAI CITY',
         district != 'MANHICA',
         district != 'MASSINGIR') %>%
  group_by(district) %>%
  tally
uw$w.weight <-
  c(0.00,
    0.22,
    0.00,
    0.19,
    0.01,
    0.00,
    0.00,
    0.27,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.07,
    0.00,
    0.00,
    0.24
    )
uw$age_group <- '0 to 4'

older <- uw
older$age_group <- '5+'
older$w.weight <- 
  c(0.00,
    0.00,
    0.00,
    0.00,
    0.10,
    0.00,
    0.00,
    0.26,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.00,
    0.35,
    0.29
)
uw <- bind_rows(uw, older)
# Overwrite with Ranjeeta's weights!
ggplot(data = uw,
       aes(x = district,
           y = w.weight)) +
  geom_bar(stat = 'identity',
           alpha = 0.6,
           fill = 'darkorange') +
  theme(axis.text.x = element_text(angle = 90)) +
  labs(x = 'District',
       y = 'Weight',
       title = 'Weights by district') +
  ggthemes::theme_hc() +
  facet_wrap(~age_group,
             nrow = 2)
```


### Comparison with other findings


Discussion

- Similar findings obtained in a cluster randomized controlled trial (RCT) estimating the effectiveness of MDA with DHA/PPQ in Zambia.

- In that study, while the number of malaria cases significantly declined after MDA in low-transmission areas, drop was not significant in areas of high transmission (ie, Magude).

- Also consistent with the predictions of a consensus study that put together 4 models focusing on the impact of MDA with a drug having similar characteristics as DHA/PPQ.  

- The latter predicted an immediate (but transient w/o other interventions) decline in malaria prevalence after MDA.  

### Quasi-experimental methods in similar contexts


Discussion


- Quasi-experimental approaches have been applied in other elimination campaigns.  

- Barofsky et al. used SCM as an approach for the identification of the impact of a malaria campaign in a district in Uganda.   
- These studies are from 50s-60s and look at economic indicators, such as primary school completion or households consumption.   
- To our knowledge, this is the first quasi-experimental evaluation of an ongoing elimination initiative using routine national surveillance data on health indicators.   

### Comparison with RCT


Discussion

- Randomized control trials (RCTs) are the gold standard for impact evaluations.   
- However, the implementation of RCTs may imply financial, practical and ethical challenges.  
- Furthermore, RCTs have debatable usefulness in identifying intervention feasibility and implementation bottlenecks, which are particularly important in disease elimination contexts.
- Quasi-experimental approaches are particularly useful when applied to routinely collected (administrative/observational) data. 

### Scaling up the method?


Discussion


- Given the focus over the last decade on malaria elimination with both large and small scale campaigns across the globe – quasi-experimental approaches offer a rigorous and inexpensive alternative to randomized-control trials. 

- Observational data is widely (and often publicly) available and true experimental approaches are often not feasible:  quasi-experimental approaches like the SCM are an excellent alternative for the evaluation of malaria elimination initiatives. 

- Approach is not only suitable for the estimation of the impact of the intervention in question, but also generalizable to malaria elimination campaigns globally.   

### Limitations 


Discussion


1. We rely on surveillance malaria cases reported at the health facility level, which is an incomplete and imperfect proxy for true malaria infection incidence since health-seeking behavior is especially low in low-income countries settings and may be different across geography and time.

2. The health facility registries from which we gathered data are themselves imperfect; we have no mechanisms by which to validate the entirety of the data. Still, BES data offers the best country-wide comparison of malaria trends at a granular level (weekly and by district).

3. Even with quasi-experimental data, we still can't fully control for abnormalities... (see next slide)

### 2017 has been anything but normal

Discussion

```{r}
x <- 
  df %>%
  group_by(year, month) %>%
  summarise(p = sum(cases) / sum(population) * 1000) %>%
  ungroup %>%
  mutate(year = factor(year))

cols <- colorRampPalette(RColorBrewer::brewer.pal(n = 9, 'Spectral'))(length(unique(x$year)))
ggplot(data = x,
       aes(x = month,
           y = p,
           group = year,
           color = year)) +
  geom_line() +
  geom_point() +
  scale_color_manual(name = 'Year',
                     values = cols) +
  labs(x = 'Month',
       y = 'Average incidence',
       title = 'Incidence by month, all districts',
       subtitle = 'Wide variability in monthly incidence') +
  theme_maltem()

```